Support xz format for template registration #11786

sudo87 · 2025-10-03T10:44:16Z

Description

This PR enables support for xz format templates #7701

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
Build/CI
Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

codecov · 2025-10-03T10:52:20Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 3.60%. Comparing base (b09f3e8) to head (1e5a903).
⚠️ Report is 3 commits behind head on main.

❗ There is a different number of reports uploaded between BASE (b09f3e8) and HEAD (1e5a903). Click for more details.

HEAD has 1 upload less than BASE

Flag BASE (b09f3e8) HEAD (1e5a903)

unittests 1 0

Additional details and impacted files

@@              Coverage Diff              @@
##               main   #11786       +/-   ##
=============================================
- Coverage     17.50%    3.60%   -13.90%     
=============================================
  Files          5894      442     -5452     
  Lines        526880    37265   -489615     
  Branches      64338     6836    -57502     
=============================================
- Hits          92243     1345    -90898     
+ Misses       424260    35757   -388503     
+ Partials      10377      163    -10214

Flag	Coverage Δ
uitests	`3.60% <ø> (-0.01%)`	⬇️
unittests	`?`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

sudo87 · 2025-10-03T11:32:30Z

@blueorangutan package

blueorangutan · 2025-10-03T11:34:04Z

@sudo87 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-10-03T12:50:36Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 15279

harikrishna-patnala · 2025-10-05T10:46:28Z

@blueorangutan test

blueorangutan · 2025-10-05T10:48:04Z

@harikrishna-patnala a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2025-10-05T11:35:46Z

[SF] Trillian Build Failed (tid-14524)

sudo87 · 2025-10-07T11:58:29Z

@blueorangutan package

blueorangutan · 2025-10-07T12:00:03Z

@sudo87 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Copilot

Pull Request Overview

Adds support for xz-compressed templates across utilities, KVM adaptors, and shell scripts to enable template registration and extraction of .xz files.

Extend compression format detection to include xz
Add extraction logic for .xz in KVM adaptors and shell scripts
Update unit tests to cover xz handling

Reviewed Changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
utils/src/main/java/com/cloud/utils/UriUtils.java	Adds "xz" to COMPRESSION_FORMATS for URL detection.
utils/src/test/java/com/cloud/utils/UriUtilsTest.java	Adds a positive assertion for xz URLs.
utils/src/test/java/com/cloud/utils/UriUtilsParametrizedTest.java	Removes ".xz" from illegal formats; tests configuration updated.
tools/devcloud4/common/development-installation/files/default/createtmplt.sh	Adds XZ detection and decompression in helper script.
scripts/storage/secondary/createvolume.sh	Adds XZ detection and decompression.
scripts/storage/secondary/createtmplt.sh	Adds XZ detection and decompression.
scripts/storage/qcow2/createvolume.sh	Adds XZ decompression.
scripts/storage/qcow2/createtmplt.sh	Adds XZ decompression.
scripts/installer/createvolume.sh	Adds XZ decompression.
scripts/installer/createtmplt.sh	Adds XZ decompression.
plugins/storage/volume/storpool/src/main/java/com/cloud/hypervisor/kvm/storage/StorPoolStorageAdaptor.java	Adds xz to extractable types and extraction command.
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/ScaleIOStorageAdaptor.java	Adds xz to extractable types and extraction command.
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/MultipathSCSIAdapterBase.java	Adds xz to extractable types and extraction command.
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/LibvirtStorageAdaptor.java	Adds xz to extractable types and extraction command.

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-10-07T12:42:38Z

...e/volume/storpool/src/main/java/com/cloud/hypervisor/kvm/storage/StorPoolStorageAdaptor.java

        } else if (downloadedTemplateFile.endsWith(".gz")) {
            return "gunzip -c " + downloadedTemplateFile + " > " + templateFile;
+        } else if (downloadedTemplateFile.endsWith(".xz")) {
+            return "xz -d -c " + downloadedTemplateFile + " > " + templateFile;
        } else {
            throw new CloudRuntimeException("Unable to extract template " + downloadedTemplateFile);
        }


isTemplateExtractable() treats ZIP as extractable, but getExtractCommandForDownloadedFile() has no branch for ".zip", which will cause ZIP files to hit the exception path. Add ZIP handling consistent with other scripts, for example: return "unzip -p " + downloadedTemplateFile + " > " + templateFile;

Copilot · 2025-10-07T12:42:38Z

...ns/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/ScaleIOStorageAdaptor.java

        } else if (downloadedTemplateFile.endsWith(".gz")) {
            return "gunzip -c " + downloadedTemplateFile + " > " + templateFile;
+        } else if (downloadedTemplateFile.endsWith(".xz")) {
+            return "xz -d -c " + downloadedTemplateFile + " > " + templateFile;
        } else {
            throw new CloudRuntimeException("Unable to extract template " + downloadedTemplateFile);
        }


ZIP is considered extractable in isTemplateExtractable(), but there is no extraction branch for ".zip" here. Add ZIP extraction support to avoid throwing for valid ZIP templates, e.g., return "unzip -p " + downloadedTemplateFile + " > " + templateFile;

Copilot · 2025-10-07T12:42:38Z

...hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/MultipathSCSIAdapterBase.java

        } else if (downloadedTemplateFile.endsWith(".gz")) {
            return "gunzip -c " + downloadedTemplateFile + " > " + templateFile;
+        } else if (downloadedTemplateFile.endsWith(".xz")) {
+            return "xz -d -c " + downloadedTemplateFile + " > " + templateFile;
        } else {
            throw new CloudRuntimeException("Unable to extract template " + downloadedTemplateFile);
        }


ZIP is allowed by isTemplateExtractable(), but ".zip" is not handled here, leading to an exception for ZIP templates. Add a ZIP case similar to other formats, e.g., return "unzip -p " + downloadedTemplateFile + " > " + templateFile;

Copilot · 2025-10-07T12:42:39Z

...ns/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/LibvirtStorageAdaptor.java

        } else if (downloadedTemplateFile.endsWith(".gz")) {
            return "gunzip -c " + downloadedTemplateFile + " > " + templateUuid;
+        } else if (downloadedTemplateFile.endsWith(".xz")) {
+            return "xz -d -c " + downloadedTemplateFile + " > " + templateUuid;
        } else {
            throw new CloudRuntimeException("Unable to extract template " + downloadedTemplateFile);
        }


ZIP archives are flagged as extractable in isTemplateExtractable(), but there is no handling for ".zip" here, which will cause a runtime exception. Add a ZIP branch, e.g., return "unzip -p " + downloadedTemplateFile + " > " + templateUuid;

vishesh92 · 2025-10-07T12:46:22Z

scripts/installer/createtmplt.sh

         ;;
  [zZ][iI][pP])  unzip -p $1 | cat > $tmpfile
        ;;
+  XZ)   xz -d -c $1 > $tmpfile


I think this is a case sensitive check. Use [xX][zZ] to match xz, XZ, xZ & Xz.

blueorangutan · 2025-10-07T13:43:26Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✖️ debian ✔️ suse15. SL-JID 15319

Support xz format for template registration

3eb9631

boring-cyborg bot added component:kvm component:primary-storage component:storage labels Oct 3, 2025

add xz command in shell script

1e5a903

vishesh92 requested a review from Copilot October 7, 2025 12:38

Copilot AI reviewed Oct 7, 2025

View reviewed changes

vishesh92 reviewed Oct 7, 2025

View reviewed changes

Support xz format for template registration #11786

Are you sure you want to change the base?

Support xz format for template registration #11786

Conversation

sudo87 commented Oct 3, 2025

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Uh oh!

codecov bot commented Oct 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

sudo87 commented Oct 3, 2025

Uh oh!

blueorangutan commented Oct 3, 2025

Uh oh!

blueorangutan commented Oct 3, 2025

Uh oh!

harikrishna-patnala commented Oct 5, 2025

Uh oh!

blueorangutan commented Oct 5, 2025

Uh oh!

blueorangutan commented Oct 5, 2025

Uh oh!

sudo87 commented Oct 7, 2025

Uh oh!

blueorangutan commented Oct 7, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Oct 7, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 7, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 7, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 7, 2025

Choose a reason for hiding this comment

Uh oh!

vishesh92 Oct 7, 2025

Choose a reason for hiding this comment

Uh oh!

blueorangutan commented Oct 7, 2025

Uh oh!

Uh oh!

codecov bot commented Oct 3, 2025 •

edited

Loading